The present invention relates to an image stabilization apparatus which is installed in an optical apparatus such as a video camera, a digital still camera, and an interchangeable lens.
Image stabilization apparatuses for detecting shakes of an optical apparatus with a shake sensor such as a gyro-sensor to reduce image shake are broadly classified into an electronic type and an optical type as disclosed in Japanese Patent Laid-Open No. 2003-219265, for example.
An important criterion to evaluate the performance of the image stabilization apparatuses is smooth behavior of the image (picked-up image) in panning. For panning control, a method of changing the cut-off frequency of a filter is used as disclosed, for example, in Japanese Patent Laid-Open No. 2003-219265.
The electronic image stabilization and the processing of panning will hereinafter be described in brief.
FIG. 10 shows the structure of an image-pickup apparatus such as a video camera in which an angular velocity sensor detects shakes and image shake correction (image stabilization) is performed by clipping a necessary pixel area from the full pixel image provided by an image-pickup element.
In FIG. 10, reference numeral 1001 shows a lens unit, and 1002 a solid-state image-pickup element such as a CCD sensor and a CMOS sensor (hereinafter referred to as a CCD). In the electronic image stabilization, the CCD 1002 has more pixels than those of a standard CCD required in a broadcast system (for example, NTSC system and PAL system). Reference numeral 1003 shows a CCD drive circuit which drives the CCD 1002 and can select which lines of the CCD are clipped as a pixel area to be output finally in a vertical synchronization direction in response to a control instruction from a microcomputer 1009, later described.
Reference numeral 1004 shows an analog signal processor which performs predetermined processing on a signal provided by the CCD 1002 to produce an analog image-pickup signal. Reference numeral 1005 shows a camera signal processor which contains an A/D converter and produces a video signal to be output finally. Reference numeral 1006 shows a line memory which can store at least one line of digital image-pickup signal through a memory control circuit 1007. The stored digital image-pickup signal can be read out from a predetermined position (address) in the line memory 1006 through the memory control circuit 1007.
The digital image-pickup signal stored in the line memory 1006 includes more pixels than those in a standard image size. The memory control circuit 1007 can select the first one of pixels to be read from the line memory 1006 in response to a control instruction from the microcomputer 1009, later described, and read the pixels corresponding to the standard image size.
Reference numeral 101 shows an angular velocity sensor which detects shakes of the image-pickup apparatus. Two angular velocity sensors 101 are generally used to detect shakes in two directions, that is, a vertical direction and a horizontal direction. Since the two sensors have exactly the same functions, only one of the directions is described herein.
Reference numeral 102 shows a high-pass filter (hereinafter referred to as an HPF) which cuts a direct-current component (hereinafter referred to as a DC component) in the output from the angular velocity sensor 101. Reference numeral 103 shows an amplifier which amplifies a detected angular velocity signal.
Reference numeral 104 shows an A/D converter which is contained in the microcomputer 1009. The angular velocity signals for the two directions are converted by this built-in A/D converter 104 into digital signals which serve as angular velocity data. The angular velocity data is subjected to predetermined signal processing by an HPF 105 and a phase compensation filter 106, passes through a variable HPF 701 which has a variable cut-off frequency, and then is input to an integrator 107. The integrator 107 generates shake correction signals for the vertical and horizontal directions based on the angular velocity data.
Reference numeral 1008 shows a correction-system controller which transmits the produced shake correction signals such that the shake correction signal for the vertical direction is transmitted to the CCD drive circuit 1003 and the shake correction signal for the horizontal direction is transmitted to the memory control circuit 1007. As described above, each of the CCD drive circuit 1003 and the memory control circuit 1007 changes the position at which the pixel area is clipped in response to the shake correction signals. When panning is detected, panning control is performed by processing such as changing the cut-off frequency of the variable HPF 701. The panning control will be described later.
Reference numeral 1009 shows the microcomputer which performs control of the CCD drive circuit 1003, the image shake correction control, and the like. The microcomputer 1009 controls the CCD drive circuit 1003 and the memory control circuit 1007 based on the shake correction signals calculated by the integrator 107 to achieve the image shake correction operation.
With the series of operations, the standard image size is extracted from the size of the full pixel image larger than the standard size, and the position at which it is extracted is controlled in response to the shake correction signals, thereby allowing correction of image shake due to camera shake or the like.
The panning control will now be described. When a user performs panning or tilting, the image is desirably changed as the user intends. However, if panning is performed during ordinary image shake correction, the image is not changed due to the image shake correction at the time of the start of panning, and the image starts to be changed abruptly when the amount of panning goes out of a range in which the image shake correction is possible. This causes the user to see a discontinuous change in the image. At the time of the end of panning, the image is fixed at the correction end (that is, a clipped pixel area or a movable member is located at the end of its movable range) and the image shake correction cannot be performed. The panning control is performed in order to avoid the situation.
As an example of the panning control, the correction-system controller 1008 detects whether or not the output from the abovementioned integrator 107 exceeds a predetermined correction amount, and if it is exceeded, the cut-off frequency of the variable HPF 701 having the variable cut-off frequency as described above is changed to remove a low-frequency signal to limit the correction amount. The panning control can restrain the image shake correction for the movement of the apparatus resulting from panning while the panning is performed, thereby achieving image shake correction operation as the user intends.
The method of changing the HPF cut-off frequency has been described as the panning control, similar control can be performed by changing the constant of integration in the integrator 107.
Besides, as disclosed in Japanese Patent Laid-Open No. H05(1993)-142615 and Japanese Patent Laid-Open No. H10(1998)-282536, a DC component may be extracted and subtracted from the output of an angular velocity sensor, or an amount equivalent to a DC component produced due to panning may be predicted and subtracted from the output of an integrator.
The panning control in the abovementioned related art, however, presents the following problems.
When the gyro-sensor is used as the shake sensor, the output from the gyro-sensor needs to be amplified by 50 to 100 times. The amplification with the high gain requires cutting of a DC component to provide a proper signal because of saturation of the output from the amplifier. For this reason, the amplification is typically performed after the DC component is cut by an HPF with a cut-off frequency of 0.05 Hz or lower.
FIG. 9 schematically shows the output from the gyro-sensor and the output from the amplifier when panning is performed with constant velocity. In reality, shakes caused by camera shake overlap with the waveforms shown in FIG. 9, but those shakes are omitted in FIG. 9.
As shown in FIG. 9, the output from the amplifier approaches the central value with time due to the effect of the time constant of the HPF described above. When the panning is ended, the voltage changed by the time constant of the HPF is generated in the reverse direction relative to the central value.
In the method of increasing the cut-off frequency of the HPF to perform the panning control described in the abovementioned related art, the cut-off frequency is returned to the ordinary value after the completion of the panning is detected. If the cut-off frequency is returned to the ordinary value simultaneously with the completion of the panning in the method, the abovementioned reverse output (voltage) causes the image shake correction to be performed in the opposite direction. This causes a swing-back phenomenon of the image, which makes the image visually undesirable.
To avoid this, the cut-off frequency needs to be held high while the reverse output is produced. The cut-off frequency during the panning is approximately 20 Hz. If the cut-off frequency is held high until the output from the HPF is stabilized after the panning is completed, a long time is taken before the image shake correction function takes effect despite the completion of the panning.
In the method of subtracting the DC component produced during the panning, there is no way to confirm whether or not the data to be subtracted is actually equivalent to the DC component produced by the panning. Thus, the swing-back may occur in the movement of the image at the completion of the panning or the image may make a different movement from an intended camera operation, leading to an unnatural image.
There is another simpler method to increase the cut-off frequency of the HPF 105 in FIG. 10. The cut-off frequency of the HPF 105 is typically set to approximately 0.1 to 0.3 Hz. The cut-off frequency can be increased to approximately 1.2 to 2.0 Hz to avoid an increase in the DC component produced during panning. As a result, the behavior of the image after the panning is improved.
In the method, however, the image stabilization performance itself is deteriorated, so that the stability of the image is reduced when an image of a still object is picked up.